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Abstract 

Portable Electronic Devices usually utilize a small screen with limited 
viewing area and a keyboard with a limited number of keys. This makes 
it difficult to perform quick searches in data arrays containing more than 
dozen items such an address book or song list. In this article we present 
a new data selection method which allows the user to quickly select an 
entry from a list using 4-way navigation device such as joystick, trackball 
or 4-way key pad. This method allows for quick navigation using just one 
hand, without looking at the screen. 
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1 Introduction 



Selecting information from lists on a mobile device is a common task, but the 
process can be quite cumbersome. A typical example of this is finding a name 
in an address book to make a phone call. If the list contains more than a dozen 
entries, scrolling through it to reach a name becomes impractical. Not all mobile 
devices have a keyboard. Navigating using virtual keyboard displayed on a touch 
screen usually requires use of a stylus. Taking out a stylus adds additional time, 
and the use of a stylus itself is relatively slow. One can use a regular phone 
keyboard to search for a name, but most selection methods offered by modern 
mobile phones are not very efficient. Also, some modern mobile phones and 
PDAs are using sliding design, where the keyboard is normally hidden and has 
to be slid out to be used (e.g. Siemens SX66). Sliding out or opening keyboard 
takes additional time. 

In this paper, we would like to present a novel approach to this problem. 
Instead of phone keyboards, a user would use a joystick, a trackball or 4-way 
key pad. Most modern mobile phones are equipped with one of these naviga- 
tion devices. In Figure 1, you can see some actual mobile phones which are 
suitable for use with the AccelKey input method. The device shown in la uses 
a trackball, lb, lc,lh use joysticks, and Id, le, lf,lg each use a four-button pad. 

The presented method is a selection method, not an input method. It does 
not allow the user to enter arbitrary data. Rather, it simply selects an entry 
from the existing dataset. [4, MacKenzie, Soukoreff (2002)] provides a good 
overview of various text input methods. 

The presented method is suitable for one-handed, "eyes- free" operation 1 . 

2 AccelKey Algorithm 

In this section, we will give a formal description of the algorithm. Please see 
Appendix A.l for a sample implementation of the algorithm in a Haskell pro- 
gramming language. 

A user is presented with a list of entries. His goal is to select one of the 
entries from the list using a 4-way navigation device. The device is capable of 
generating at least five events triggered by user actions. Directional events up, 
down, left, and right are generated when the joystick is tilted or the trackball 
is rotated in the appropriate direction. An additional select event is generated 
when the user pushes the joystick or trackball down, or presses a dedicated 
button. Additional events, such as backspace or reset could be assigned to to 
further keys to improve user experience, but are not strictly necessary for this 
algorithm. They are covered in Section 3.1. 

Each directional event is associated with a group of letters from the alphabet 
from which elements of the list are composed. We will call such assignments a 
"Layout" . Issues related to layout selection will be covered in the Section 2.4. 
For now, let us just assume the letter assignment is non-overlapping i.e., each 

1 Using focus of attention notation[l], this method could be classified as single FOA task. 
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letter is associated only with one of four directional events. Not all characters of 
the alphabet have to be associated with directional events. Characters which are 
not associated with events (e.g punctuation marks and whitespace) are deemed 
insignificant and ignored by the algorithm. 

The algorithm could be in one or two modes: selection or scrolling. The 
initial state is selection. 

2.1 Selection Mode 

When in this mode, the algorithm maintains a sequence of directional events 
generated by the user. We will call this sequence a prefix. Initially, the prefix 
is empty Each new directional event generated by the user is appended to the 
end of the prefix. The list is then filtered by excluding all entries except ones 
matching the current prefix. The entry is matching the prefix if, and only if in 
the current layout each of its letters belongs to the group associated with the 
prefix event with the same index. 2 

At any moment the user can chose to generate a select event. When this 
happens, if the current, filtered list contains only one element the selection 
process is completed. If the list contains more than one element, we switch to 
the scrolling mode described in Section 2.2 to narrow down the selection further. 

This mode can be likened to presenting a user with a keyboard with 4 keys: 
the four keys have groups of letters on them, and there is an additional select 
key. The user types his search term using this keboard, and the algorithm 
performs disambiguation using list content as a dictionary. In some respects 
this is similar to the Tegic T9® input method. 

2.2 Scrolling Mode 

Upon entering this mode one of the list elements is selected as current. 3 Using 
up and down events, the user then navigates towards the entry he is looking for 
and selects it using the select event. Directional events up and down move the 
selection pointer, selecting previous or next list elements as current. Any events 
of left or right will bring the algorithm back to selection state, using the last 
used prefix. Finally, select will indicate that the current element is the ultimate 
user selection. 

2 Case-sensitive or case-insensitive check could be performed, depending on the nature of 
the data presented in the list. 

3 Initial selection of a current element is not as simple as it seems. Though the first entry 
might seem to be an obvious choice, selecting the element in the middle of the list instead 
would minimize the average required number or keystrokes. If the list is too long to fit on the 
screen, the sort order becomes important, because the user should be able to decide in which 
direction to scroll based on information he or she sees on the screen. For this, the list must 
be sorted. The sorting order and criteria are not important for the algorithm, as long as it 
obvious to the user in which direction to scroll from the current element to reach the element 
he is looking for. If the first element is selected initially, the sorting requirement can be lifted 
since the only direction the user can scroll is down. 
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2.3 Comparison to Other Methods 

In this section, we will compare AccelKey to a few other selection algorithms. 
The algorithms we have chosen to evaluate are: 

Scroll - This is the simplest possible algorithm. A user is presented with the 
list and a cursor. The cursor initially points to the first clement. Using up and 
down arrows, a user can select the desired list element. 

Multi-Tap First - This is the only selection algorithm which is implemented 
in some mobile phones (for example Motorola V600) besides Scroll algorithm. A 
user can select the first letter of the name using the multi-tap method[8]. Once 
the first letter is selected, the cursor is positioned to the first entry starting with 
this letter and then, user can scroll down to select a desired name using scroll 
method. 

Mult i- Tap Match - This is a more advanced method, implemented in some 
phones (e.g. Sony Ericsson K750) . A user can type part of the name using multi- 
tap input method. As he types the cursor moves to the first name that matches 
the typed prefix. At any moment the user can switch to the scroll method 
to continue selection. In our evaluation, we implemented the best possible 
strategy, deciding when it was the best time to switch from typing to scrolling. 
We suspect most users might not perform as well as our program, but we will 
use these numbers for an evaluation, as a best case scenario. To compare the 
efficiency of these algorithms we took several datasets and ran all algorithms 
on each of them. The measure of efficiency was an average number of events 
(user actions such as key presses or joystick movements) required to select any 
name from the dataset. The evaluation was performed using a program which is 
included in the Appendix A. The AccelKey algorithm used a QWERTY layout 
(see Section 2.4). The results are shown in the Table 1. 





Method 


Dataset 


AccelKey 


Scroll 


Multi-Tap 


Multi-Tap First 


Writers 


4.08 


47.5 


4.66 


4.67 


Representatives 


5.16 


196.5 


6.21 


6.22 


Graduates 


7.11 


684 


7.58 


7.58 



Table 1 : Selection Methods Efficiency Evaluation (average number of events per 
entry) 

Following datasets were used: 

Writers This dataset consists of the "100 Best Writers of the 20th Century" [5] . 
It contains 96 entries. Only last names were used. 
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Representatives The dataset is a list of names of members of U.S. House of 
Representatives.^)] It contains 394 entries. Only last names were used. 

Graduates The dataset is a list of names of students to receive degrees at 
University of New Brunswick in May 2004. [7] The list contains 1369 entries. 
Only last names were used. 

As we can see, on all three datasets AccelKey algorithm requires fewer 
keystrokes than others. Also, since it is usually can be operated without re- 
moving one's finger from the joystick, there is no time cost of moving fingers 
between keys. 

2.4 Layout Evaluation 

Selected layout (the association between letters and directional events) can sub- 
stantially affect the performance of AccelKey. There are a few approaches to 
optimize layout: 

Mnemonically - to make the layout easy to remember. One obvious layout 
is alphabetical. Another possible layout is based on the QWERTY keyboard [ ] 
layout. Experienced computer users may find it easier to use than alphabetical, 
since usually one can remember at least the general location of keys on his or 
her keyboard. 

Statistically - to minimize the number of events required to select any list 
element. Here, the layout would depend on a corpus: letter frequencies, prob- 
abilities of one letter following another, etc. Layout optimization algorithms is 
one of the subject of our future research. Using source code from Appendix A 
we have evaluated two layouts: 

ABC Layout splits the Latin alphabet in its traditional order into 4 groups: 
[A,B,C,D,E,F,G](up), [H,I,J,K,L,M,N] (left), [0,P,Q,R,S,T,U] (right), [V,W,X,Y,Z] 
(down). 

QWERTY Layout rougly emulates key positions on a personal computer 
keyboard (QWERTY layout[ ], US version), arranging letters into four groups: 
[Q,W,E,R,T,Y,U,I,0,P] (up), [A,S,D,F,G] (left), [H,J,K,L] (right), [Z,X,C,V,B,N,M] 
(down). 

Using datasets used in previous section, the average number of events to 
select any dataset element using a given layout is shown in Table 2. 

As we can see, on the Writers and Graduates datasets, an ABC layout 
has shown to give better results, while QWERTY layout fared better with the 
Representatives dataset. 
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Layout 


Dataset 


QWERTY 


ABC 


Writers 


4.08 


4.07 


Representatives 


5.16 


5.24 


Graduates 


7.11 


6.76 



Table 2: Layout Efficiency (average number of events per entry) 

3 Algorithm Extensions 

3.1 Additional Events 

For user convenience, we can introduce some additional events. These events 
are not essential to the core algorithm operation, and their implementation is 
optional. Backspace - eliminates effects of the previous event and brings the 
algorithm to the previous state. Reset - resets the algorithm to the initial state, 
which is scrolling mode, with an empty prefix. 

3.2 Using Keyboard or KeyPad 

On devices equipped with a keyboard in addtition to a 4-directional input device, 
AccclKcy selection algorithm could be combined with more traditional input 
methods. 

For example, on mobile phones equipped with a standard phone numeric 
keyboard, numbered keys could be generating events which are treated as an 
additional nine events using special layout: [A,B,C] (Key "2"), [D,E,F] (Key 
"3"), [G,H,J] (Key TO, [J,K,L] (Key "5"), [M,N,0] (Key "6"), [P,Q,R,S] (Key 
"7"), [T,U,V] (Key "8"), [W,X,Y,Z] (Key "9"). Thus, the user could mix 
directional events from a joystick and keypad events during the selection process. 

On devices equipped with a QWERTY (or similar) keyboard, users should 
be able to combine AccelKey directional events with keypreses from a keyboard 
during the selection process. For example, tilting a joystick upwards and then 
pressing the "z" key on a keyboard would select all entries which start from any 
letter associated with the up event in the current layout, followed by the letter 



3.3 Using trackball 

It is possible to use AccelKey with devices equipped with a trackball (for exam- 
ple, BlackBerry Pearl). Unlike a joystick which generates descrete directional 
events, a trackball generates events containing the following values: 

Ax Magnitude of navigational motion: negative for a move left and postive for 
a move right. 
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Ay Magnitude of navigational motion: negative for an downwards move, and 
positive for a upwards. 

Mapping between these values and directional events used by AccelKey al- 
gorithm could be expessed as: 

right , if |Ax| > \Ay\ and Ax > 0, 
left , if |Ai| > \Ay\ and Ax < 0, 
up , if |Ax| < \Ay\ and Ay > 0, 
down , if |Ax| < \Ay\ and Ax < 0, 

This mapping for Ax and Ax in [—10, 10] interval is shown in Figure 2. 




-10 -5 5 10 

Ox 



Figure 2: Mapping trackball movement offsets to directional events 

Some trackball devices are very sensitive and report even very slight move- 
ments, which makes it necessary to implement jitter elimination. The simplest 
form is to ignore trackball events with movement amplitude less than a certain 
value. In other words, ignore trackball events if \J Ax 2 + Ay 2 is less than some 
pre-defined small value. 



event(Ax, Ay) 
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3.4 Multiple Word Selection 

The algorithm could also be extended to handle lists with multiple words per 
entry. One example of such a list is an address book, where for each entry you 
can commonly have a combination of first, last, and middle names. 

The most obvious approach here is to simultaneously match the prefix to 
the beginning of the first, last and middle names, thus allowing a user to select 
using any of these fields. For example, to select Jorge Luis Borges the user can 
enter Jorge or Luis or Borges 4 . 

Though a multiple word match approach is useful, it could be further im- 
proved upon. For example, selecting a prefix or matching a common first name 
presents us with a fairly long list of choices which cannot be narrowed down 
further without switching to a selection mode. One example of such a case is 
shown in Figure 3. 



T 


AtcelKev 


"«1 








s 


John, Steinbeck 
John Cheever 
John Steinbeck 
John Grisham 
John Irving 
John HcPhee 






ZXCVBNM 




Options Exit 



Figure 3: Example of an address book selection using common first name 

To solve this, we can allow for prefix matching to span boundaries of adjacent 
words. Thus, in the example shown in Figure 3, the user should be able to 
generate events matching letters: "j" , "h" , "o" , "n" , "u" to narrow the list 
down to single entry: "John Updike". Of course, prefix match could span 
more than two words, as long as all of them match sequentially (for example 
"Arthur,C, Clarke" could be matched as "a", "r", "t", "h", "u", "r", "c", "c"). 

In some applications, we might want to allow matching to "wrap" at the 
end of the last word in a sequence and to proceed from the beginning of first 
word. For example, this would allow us to match "Smith, John" as "j", "h", 
"o" , "n" , "s" . If matching is always to stop at the last word in a group, there 
is a no way to disambiguate "Smith, John" from "Blake, John" after the first 
name was matched. 

4 This could be extended to match as many words as needed, for example to use full name 
like Jorge Francisco Isidoro Luis Borges Acevedo. 
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4 Implementation 



The algorithm was implemented as a Java Applet, BlackBerry application, 
J2ME MIDlet and SymbianOS application, and has been tested on several mo- 
bile phones. The latest demo version can be downloaded from the following 
URL: http://www.accelkey.com/ 

The AccelKey application is shown running on a mobile phone in Figure 4. 

Let us examine the screen in more detail. In the middle of the screen is 
the list of names from which we will be selecting. They are displayed in First, 
Middle, Last Name format. As a user sends more events using the joystick, 
the content of this list is narrowed down to contain only names matching the 
current prefix. In each name, the letters matching the current prefix are shown 
in red. During selection, we ignore all punctuation marks, and other symbols 
not represented in the current layout. Thus, to select Thomas Stearns Eliot, 
abbreviated in our list as T.S., Eliot, a user must generate events for letters 
"T" (up) and "S"(Ze/t), omitting punctuation marks and spaces. On the top, 
bottom, and sides of the list, we can see groups of letters displayed. This is a 
visual aid, showing the current layout. The groups are arranged in accordance 
with the direction in which a user must tilt the joystick to generate an event 
for this group. Some of these letters are grayed out. That means that in the 
current list, with given selection prefix, there are no entries which contain this 
letter as the next choice. Graying them out helps a user to find the next letter 
more easily. The AccelKey selection method could be used with languages using 
alphabets besides the Latin. For example on Figure 5 we can see an example 
using the Cyrillic alphabet. 

5 Further Work 

One direction for future research is the development of algorithms which would 
help to minimize the average number of events per list element, thereby opti- 
mizing layout. 

Another interesting direction to explore is the use of touch-sensitive screens 
with this algorithm. It should be possible to use "gestures" performed by the 
user touching the screen as an input method. 

On the implementation side, we would like to experiment further with dif- 
ferent screen arrangements and visual and possibly audible feedback to users 
during selection process. A trackball jitter elimination algorithm could also be 
further fine tuned. 

A more precise comparative evaluation of the alogrithm efficiency using 
Hicks-Hayman[ , 2] and Fitts[3] laws is also in our plans. 
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Figure 5: AccelKey with Cyrillic Alphabet 

A Source Code 

A.l Fast4.hs 

{- 

Fast4 Algorithm Implementation 

(c)2007 Vadim Zaliva <lord@crocodile . org> 

-} 

module Fast4(f 4f ilter , eventSeq, countEvents) where 

import List 
import Maybe 

— I Definition of layout - a mapping of 4 directional 

— events to group of characters 
f 41ayout : : [ [Char] ] 

{- 

f41ayout = ["qwertyuiop" , 
"asdfg" , 
"hjkl" , 
"zxcvbnm"] 

-} 

f41ayout = ["abedefg", 
"hijklmn" , 
"opqrstu" , 
"vwxyz"] 
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— I A complete alphabet used in current layout 
f 4abc : : [Char] 

f4abc = ((nub . concat) f41ayout) 

— I Checks if given prefix (events sequence) match given sting 
f4match : : [Int] -> String -> Bool 

f4match [] _ = True 
f4match _ [] = False 

f4match (p:px) (s:sx) = if elem s f4abc then 

(elem s (f41ayout ! !p) ) && (f4match px sx) 
else f4match (p:px) sx 

— I Filter list, removing all elements which do not match given prefix 
f4f ilter : : [Int] -> [String] -> [String] 

f4filter p 1 = filter (f4match p) 1 



— Followg code is used only for metric calculation — 

— I Given list of strings, will count number of events required to 

— select each of them from this list using Fast 4 Algorighm 
countEvents : : [String] -> [Int] 

countEvents 1 = map (\x -> eventSeq 1 [] x x) 1 

eventSeq : : [String] -> [Int] -> String -> String -> Int 
eventSeq 1 p [] s = length p + scrollSeq 1 s 
eventSeq 1 p (x:xs) s = min (length p + 1 + scrollSeq 1 s) ( 

if elem x f4abc then 

eventSeq (f4filter np 1) np xs s 
else eventSeq 1 p xs s) 
where 

np = p ++ [findEvent x] 

— I Returns number of events required to select item from the list in 

— "scroll" mode. Initially pointer is positioned in the middle of the 

— list 

scrollSeq: : [String] -> String -> Int 

scrollSeq 1 x = abs (fromMaybe (findlndex (cmp x) 1) - (div (length 1) 2)) 
where 

cmp a b = (intersect a f4abc) == (intersect b f4abc) 

— I Returns event number associated with given char in current layout 
findEvent : : Char -> Int 

findEvent x = fromMaybe (findlndex (elem x) f41ayout) 
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A. 2 MultiTap.hs 

{- 

Multitap Search Algorithm Implementation 
(c)2007 Vadim Zaliva <lord@crocodile . org> 

-} 

module MultiTap(countTaps, countTapAndScrolls) where 

import List 
import Maybe 
import Scroll 

— I Definition of layout - a mapping of 4 directional 

— events to group of characters 
mt layout : : [[Char]] 

mtlayout = ["abc", 
"def", 
"ghi", 
"jkl", 
"mno" , 
"pqrs", 
"tuv" , 
"wxyz" 

] 

— I A complete alphabet used in current layout 
mtabc : : [Char] 

mtabc = ((nub . concat) mtlayout) 

— I Given list of strings, will count number of events required to 

— select each of them from this list using MultiTap filtering 
countTaps : : [String] -> [Int] 

countTaps 1 = map (tapSeq If) If 
where 

If = map (intersect mtabc) 1 

— I Given list of strings, will count number of events required to 

— select each of them from this list using MultiTap + scroll filtering 
countTapAndScrolls : : [String] -> [Int] 

countTapAndScrolls 1 = [tapCost x + tapSeq [is I (i : is)<-lf , i==x] xs I (x:xs)<-lf] 
where 

If = map (intersect mtabc) 1 

tapSeq : : [String] -> String -> Int 
tapSeq _ [] = 

tapSeq 1 (x:xs) = min (scrollDistance 1 (x:xs)) 

((tapCost x) + (tapSeq [is I (i : is)<-l , i==x] xs)) 
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tapCost : : Char -> Int 

tapCost x = 1 + fromMaybe (elemlndex x (fromMaybe "" (find (elem x) mtlayout))) 



A.3 Scroll.hs 

{- 

Scroll-to-select algorithms metrics caluclation 
(c)2007 Vadim Zaliva <lord@crocodile . org> 

-} 

module Scroll (countScrolls , scrollDistance) where 

import Maybe 
import List 

countScrolls : : [String] -> [Int] 
countScrolls 1 = map (scrollDistance 1) 1 

— I Returns number of events required to select item from the list in 

— "scroll" mode. Initially pointer is positioned in the beginning of the 

— list 

scrollDistance : : [String] -> String -> Int 
scrollDistance 1 x = fromMaybe (elemlndex x 1) 
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